03a-TIMCI-SPA-fa Audit Analysis
Lucas Silbernagel
23 September, 2021
library(dplyr)
library(readr)
library(ggplot2)
library(openxlsx)
library(knitr)
library(tibble)
library(stringr)
library(stringi)
library(readxl)
library(lubridate)
library(shiny)
library(plotly)1 Loading the Data and Removal of Training Data
# Unzip and extract ODK data from ODK zip
df <- as.data.frame(extract_data_from_odk_zip(params$file_path_zip, params$file_name_csv))
# Formatting dates from integer (in ms) to time stamp
df$start <- format_date_ms(df$start)
df$end <- format_date_ms(df$end)
# filtering for events that occurred after 18th July 21
df <- subset(df, as.Date(start) > as.Date("18.07.2021", "%d.%m.%Y"))2 Deriving New Features
2.1 Time Spent per Event
# subtracting end from start date
df$time_spent = round(as.numeric(df$end - df$start))2.2 Question
# splitting the node strings so that only the question name remains
df$question = sapply(df$node, create_question)2.3 Question Decoded
df <- decode_question(df, df$question, params$codebook)2.4 Categorical Answers Decoded
df <- decode_categories(df, params$codebook)2.5 Time until a Response was Changed + Stream of Answer Changes
df <- df %>%
# bringing the data in the right order
arrange(`instance ID`, node, start) %>%
# adding two empty columns to store the new features in
add_column(time_till_change=NA) %>%
add_column(changed_from=NA)
# iterating over the df and computing the time it took until an answer was changed + adding what the question was before
for (i in 1:nrow(df)){
if (df$`old-value`[i]==df$`new-value`[i-1] && !is.na(df$`old-value`[i]) && !is.na(df$`new-value`[i-1]) ){
df$time_till_change[i] <- round(as.numeric(df$start[i]-df$end[i-1]))
} else{
next
}
}2.6 Preview and Summary of the Final Data
head(df)| instance ID | event | node | start | end | latitude | longitude | accuracy | old-value | new-value | time_spent | question | question_decoded | new_value_decoded | old_value_decoded | time_till_change | changed_from |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| uuid:277e3be0-ea5a-44ec-b869-4e0c56b13c8d | group questions | /data/b2 | 2021-08-10 08:23:55 | 2021-08-10 12:52:13 | NA | NA | NA | NA | NA | 16098 | b2 | NA | NA | NA | NA | NA |
| uuid:277e3be0-ea5a-44ec-b869-4e0c56b13c8d | question | /data/b2/b2_3 | 2021-08-10 08:23:55 | 2021-08-10 12:52:13 | NA | NA | NA | NA | 2, 3 | 16098 | b2_3 | In addition to outpatient services, what other services are provided in the facility? | Daytime observation capacity, Emergency department | NA | NA | NA |
| uuid:277e3be0-ea5a-44ec-b869-4e0c56b13c8d | question | /data/b2/b4_1 | 2021-08-10 08:23:55 | 2021-08-10 12:52:13 | NA | NA | NA | NA | 1, 3, 5, 2, 4 | 16098 | b4_1 | Opening days | Monday, Wednesday, Friday, Tuesday, Thursday | NA | NA | NA |
| uuid:277e3be0-ea5a-44ec-b869-4e0c56b13c8d | question | /data/b2/b4_2 | 2021-08-10 08:23:55 | 2021-08-10 12:52:13 | NA | NA | NA | NA | 2 | 16098 | b4_2 | Daily opening hours | 5 to 8 hours | NA | NA | NA |
| uuid:277e3be0-ea5a-44ec-b869-4e0c56b13c8d | question | /data/b2/b4_3 | 2021-08-10 08:23:55 | 2021-08-10 12:52:13 | NA | NA | NA | NA | 0 | 16098 | b4_3 | At any time during the last 7 days has the facility been unexpectedly closed? | No | NA | NA | NA |
| uuid:277e3be0-ea5a-44ec-b869-4e0c56b13c8d | question | /data/b2/d1_a_1 | 2021-08-10 08:23:55 | 2021-08-10 12:52:13 | NA | NA | NA | NA | 1, 2, 3, 4 | 16098 | d1_a_1 | Services provided to children 2-59 months | Vaccination, Growth monitoring, Nutrition, Outpatient curative services | NA | NA | NA |
summary(df)## instance ID event node start
## Length:3150 Length:3150 Length:3150 Min. :2021-07-19 09:22:16
## Class :character Class :character Class :character 1st Qu.:2021-07-28 12:17:32
## Mode :character Mode :character Mode :character Median :2021-08-10 13:03:05
## Mean :2021-08-06 15:03:51
## 3rd Qu.:2021-08-17 14:08:02
## Max. :2021-08-30 14:57:34
##
## end latitude longitude accuracy
## Min. :2021-07-19 09:22:19 Mode:logical Mode:logical Mode:logical
## 1st Qu.:2021-07-28 12:17:40 NA's:3150 NA's:3150 NA's:3150
## Median :2021-08-10 13:07:18
## Mean :2021-08-06 15:34:50
## 3rd Qu.:2021-08-17 14:13:21
## Max. :2021-08-30 14:57:34
## NA's :162
## old-value new-value time_spent question
## Length:3150 Length:3150 Min. : 0.0 Length:3150
## Class :character Class :character 1st Qu.: 101.0 Class :character
## Mode :character Mode :character Median : 259.0 Mode :character
## Mean : 523.2
## 3rd Qu.: 520.0
## Max. :67044.0
## NA's :162
## question_decoded new_value_decoded old_value_decoded time_till_change
## Length:3150 Length:3150 Length:3150 Min. :-18.000
## Class :character Class :character Class :character 1st Qu.: 2.000
## Mode :character Mode :character Mode :character Median : 3.000
## Mean : 4.137
## 3rd Qu.: 3.000
## Max. : 49.000
## NA's :2989
## changed_from
## Mode:logical
## NA's:3150
##
##
##
##
## 3 General Information about the Data
no_inst = length(unique(df$`instance ID`))
no_event = nrow(df)
earliest_start = as.Date(min(df$start))
latest_end = as.Date(max(df$end[!is.na(df$end)]))Total number of instances: 14
Total number of events/questions: 3150
Examination period: 2021-07-19 - 2021-08-30
4 Grouped by Time
4.1 Events/Questions Started by Day
df_by_day <- df %>%
mutate(start_date = as.Date(start)) %>%
count(start_date, name = "count")
gg1 <- ggplot(df_by_day, aes(x = start_date, y = count)) +
geom_line() +
geom_smooth(alpha=0.5, colour="red", method="loess", se=F) +
labs(title = "Number of Events/Questions Started by Day with Smoothed Regression Line", y = "Number of Questions/Events Started", x = "Satrt Date") +
theme_light()
gg1
4.2 Questions/Events started by Weekday and Hour of the Day
df_wday_hour <- df %>%
mutate(wday=wday(start, label=T, week_start = 1), hour=hour(start)) %>%
count(wday, hour, name="count_wday_hour") %>%
arrange(desc(wday))
theme_heatmap <- theme_light() +
theme(panel.grid = element_blank(),
panel.border = element_blank(),
plot.title = element_text(face = "bold", size = 11, hjust = 0.5),
axis.ticks = element_blank(),
axis.title.x = element_blank(),
axis.title.y = element_text(size=10),
axis.text.y = element_text(size = 8),
axis.text.x = element_text(size = 10),
legend.position = "none")
gg2 <- ggplot(df_wday_hour, aes(x = wday, y = hour, fill = count_wday_hour)) +
geom_tile(colour="white") +
scale_fill_gradient(low = "#fff0f0", high="#940606") +
scale_y_reverse(breaks=c(23:0), labels=c(23:0), expand = c(0,0)) +
scale_x_discrete(expand = c(0,0), position = "top") +
labs(title = "Number of Started Events/Questions by Day of Week / Hour of Day", y = "Hour of Day") +
geom_text(aes(label = count_wday_hour), size = 2) +
theme_heatmap
gg2
4.3 Distribution of Time Spent per Event/Question with largest 5 % removed
df_clean = subset(df, time_spent<quantile(df$time_spent,0.95, na.rm=TRUE))
hist(df_clean$time_spent[!is.na(df_clean$time_spent)]/60, breaks=20, xlab = "Time Spent in Minutes", main = "Histogram of the Time Spent by Question")
5 Aggregated by Event/Question
5.1 Median Time Spent by Question
df_median_time_per_question <- df %>%
filter(event=="question") %>%
group_by(question_decoded) %>%
summarise(median_time_spent = median(time_spent)) %>%
arrange(desc(median_time_spent)) %>%
mutate(median_time_spent = round(seconds_to_period(median_time_spent)))
df_median_time_per_question| question_decoded | median_time_spent |
|---|---|
| How are areas for quality improvement in relation to clinical service delivery identified / agreed? | 29M 22S |
| How are possible solutions to identified areas for quality improvement discussed? | 29M 22S |
| How is implementation of solutions to identified areas for quality improvement supervised? | 29M 22S |
| How is implementation of solutions to identified quality improvement needs monitored? | 24M 42S |
| Toilet functional | 21M 29S |
| What happens when the internet monthly data limit is reached? | 21M 29S |
| What did the supervisor observe during the last visit? | 21M 4S |
| Who provided the external supervision? | 20M 4S |
| What quality improvement goals or initiatives in relation to clinical service delivery are currently ongoing in this facility? | 19M 36S |
| Did the supervision visit result in the following outputs? | 19M 16S |
| How many external supervision visits have occurred during the 12 last months? | 19M 16S |
| How many healthcare providers receive supportive supervision? | 19M 16S |
| When was the last time this facility received an external supervision visit? | 19M 16S |
| Was IMCI part of the supervision visit? | 17M 46S |
| What did the supervisor do during the last visit? | 17M 46S |
| Is the emergency department separate from the outpatient area? | 17M 28S |
| Triage for children 0-59 months | 17M 28S |
| How do external supervisors monitor the quality of services at this facility? | 16M 15S |
| What did the supervisor in terms of training / support during the last visit? | 14M 52S |
| What did the supervisor review during the last visit? | 14M 52S |
| For this health facility, who decides which services are available/on offer to clients? | 13M 23S |
| Days per week facility for which there is normally at least one eligible healthcare professional is available to consult children 0-59 months? | 13M 22S |
| Does someone else consult if the eligible provider is unavailable? | 13M 22S |
| Number of days for which there was no eligible healthcare provider available to consult children 0-59 months in the last 7 days? | 13M 22S |
| Total number of consultation rooms available for simultaneous consultation of children | 13M 22S |
| Did the last supervision visit result in the planning of the following activities? | 13M 12S |
| Is there a system at the facility to keep track of staff trainings and certifications? | 13M 10S |
| When was the last time the facility committee happened? | 12M 56S |
| Who is responsible for the day to day management of the facility? | 12M 56S |
| How frequently is data quality checked? | 12M 42S |
| For this health facility, who participates in the budget planning and budget development process? | 11M 32S |
| Source of supplies / consumables / equipment for this facility? | 11M 32S |
| What is the procedure if a caregiver is unable to pay for any of the fees associated with healthcare provided to a sick child 0-59 months in this facility? | 11M 32S |
| Where activities were identified, were they followed upon as part of supervision? | 11M 16S |
| Does this facility receive any external supervision? | 11M 6S |
| Does this health facility have any current goals or initiatives for quality improvement in relation to clinical service delivery? | 11M 6S |
| Frequency of facility committees | 11M 6S |
| Are fees charged for the following services for healthcare provided to a sick child 0-59 months ? | 10M 8S |
| Do the funds or resources available to the facility depend on the facility’s past or current performance against specific metrics or goals (either overall or for specific designated targets or goals)? | 10M 8S |
| For this health facility, is the budget based on categories of types of expenses (such as staffing, medicines, equipment etc.) or based on categories of the types of services (eg. HIV services, MCH services etc.), or both? | 10M 8S |
| For this health facility, who makes decisions about the overall staffing structure and staffing resources (how many providers and staff are hired to work here, the types of staffs are hired to work here etc.)? | 10M 8S |
| Is there any mechanism to ensure the accuracy of recorded information on registries? | |
| OBTAIN documents related to data quality checks (reports, feedback, checklists). | 10M 3S |
| For what population is the facility policy to charge no fees? | 9M 60S |
| For what kind of trainings are these mechanisms in place? | 9M 51S |
| Training log management system | 9M 51S |
| Does this facility have budget earmarked for staff specifically hired for and designated to provide services for infants and/or children exclusively (and not adults)? | 9M 42S |
| For this facility, is there a budget or allocation of funds specifically designated to be used for services, staffing, medicines or equipment for infant and or child health curative services (and not other services such as PMTCT, HIV care and treatment or adult services)? | 9M 42S |
| Source of revenue or funding of this facility? | 9M 42S |
| How much time is taken from the facility staff to report facility data related to outpatient children 0-59 months per week? | 9M 17S |
| Individual clinical notes | 9M 17S |
| New visits reported to HMIS? | 9M 17S |
| Referrals reported to HMIS? | 9M 17S |
| Registers / ledger recording available at the facility | 9M 17S |
| Diagnoses reported to HMIS? | 8M 54S |
| Number of sick children per month reported to HMIS | 8M 54S |
| Total number of children under 5 consultations reported to HMIS? | 8M 54S |
| Total number of subgroup of # young infant/neonatal consultations reported to HMIS | 8M 54S |
| Vaccination / monitoring area / room separate from curative services | 8M 46S |
| Is there an “observation area” where children and young infants can stay for longer observation without admission? | 8M 35S |
| On average, hours per day facility at least one eligible healthcare professional is available to consult children 0-59 months? | 8M 35S |
| Communicable disease services | 8M 34S |
| Deliveries at the facility (sub-categories if C-section vs other available) reported to HMIS? | 8M 32S |
| Follow-up visits reported to HMIS? | 8M 32S |
| At any time during the last 7 days has the facility been unexpectedly closed? | 8M 30S |
| Daily opening hours | 8M 30S |
| In addition to outpatient services, what other services are provided in the facility? | 8M 30S |
| Opening days | 8M 30S |
| Recommendations on alternative transports | 8M 30S |
| Surgical services (other than C-Section) | 8M 30S |
| Who organises transport to referral facility for children 0-59 months? | 8M 30S |
| Services provided to children 2-59 months | 8M 26S |
| Among registers available at the facility, which one(s) are used for HMIS outpatient children 0-59 months reporting? | 8M 22S |
| Segregation of waste | 7M 46S |
| Check completeness (e.g. blank counts) of (provisional or final) diagnoses over the last 20 entries | 7M 44S |
| Check readability of (provisional or final) diagnoses over the last 20 entries | 7M 44S |
| Check relevance of (provisional or final) diagnoses over the last 20 entries | |
| (e.g. check diagnoses only, not symptoms are reported as diagnoses) | 7M 44S |
| Laboratory register / ledger recording specific for children 0-59 months | 7M 44S |
| Pharmacy records management system? | 7M 44S |
| Pharmacy register / ledger recording specific for children 0-59 months | 7M 44S |
| Clinical observations recorded in the registration or consultation ledger | 7M 33S |
| Laboratory records management system? | 7M 33S |
| Are tablets provided by the government at the facility? | 7M 26S |
| Available modes of communication | 7M 26S |
| Hand-washing soap / liquid soap | 7M 26S |
| Main electricity source | 7M 26S |
| Most commonly used source of water for the facility at this time? | 7M 26S |
| running_water | 7M 26S |
| Toilet in the facility for outpatient use | 7M 26S |
| Diagnoses recorded in the consultation ledger | 7M 22S |
| Alcohol based handrub | 7M 5S |
| Are computers provided by the government at the facility? | 7M 5S |
| Personal protective equipment | 7M 5S |
| Days of the week for which there are dedicated vaccination / growth monitoring services | 6M 57S |
| Please specify (d1.2) Back up electricity source(s)) | 6M 4S |
| Salbutamol | 6M 4S |
| Stock outs in the last 7 days | 6M 4S |
| Among those services, which ones are also provided to children 0-2 months? | 6M 3S |
| Artesunate (rectal) | 6M 1S |
| Cloxacillin | 6M 1S |
| Co-trimoxazole syrup / suspension | 6M 1S |
| Gentian violet | 6M 1S |
| IV fluid - glucose | 6M 1S |
| IV fluid - ringer’s lactate / sodium chloride | 6M 1S |
| Mebendazole Tablet | 6M 1S |
| Neonatal IV needles | 6M 1S |
| Nitrofurantoin | 6M 1S |
| Oral artemisinin combination therapy (paediatric dose) | 6M 1S |
| Paediatric IV needles | 6M 1S |
| Paracetamol syrup/suspension | 6M 1S |
| Primaquine | 6M 1S |
| Rifampicin / Isoniazid / Pyrazinamide / Etambutol (RHZE) | 6M 1S |
| Diazepam | 5M 57S |
| NG tubes | 5M 57S |
| Quinine (oral) | 5M 57S |
| Artemether injection | 5M 53S |
| Chlorhexidine | 5M 53S |
| Tasks for which computers are used? | 5M 48S |
| Artesunate (injection) | 5M 44S |
| Cups/palladium for neonatal feeding | 5M 44S |
| IV giving set | 5M 44S |
| Metronidazole | 5M 44S |
| Oral Rehydration Salts (ORS) sachets | 5M 44S |
| Quinine (injection) | 5M 44S |
| Rifampicin / Isoniazid (RH) | 5M 44S |
| Stock stored in first-to-expire, first-out (FEFO) order (i.e. the stock that will expire first is the closest to the front) | 5M 44S |
| Vitamin A (retinol) capsules | 5M 44S |
| Zinc sulphate syrup or dispersible tablets | 5M 44S |
| NA | 5M 44S |
| Azithromycin | 5M 40S |
| Gentamicin | 5M 40S |
| Stock replenishment mechanism | 5M 40S |
| Amoxicillin - dispersible tablets (250mg) | 5M 28S |
| Amoxicillin - dispersible tablets (500mg) | 5M 28S |
| Amoxicillin - syrup / suspension (250 mg) | 5M 28S |
| Ampicillin | 5M 28S |
| Benzylpenicillin | 5M 28S |
| Ciprofloxacin | 5M 28S |
| Co-amoxiclav (Amoxicillin + Clavulanic acid) | 5M 28S |
| Amoxicillin - syrup / suspension (125 mg) | 5M 20S |
| Ceftriaxone | 5M 20S |
| Is the internet network provided by government accessible today? | 4M 53S |
| Monthly data limit of internet provided by government at the facility | 4M 53S |
| Number of computers seen in use during the assessment | 4M 53S |
| What is the mobile phone network signal strength today (bar level on the respondent’s phone)? | 4M 53S |
| Where do you refer children 0-59 months who require oxygen then? | 4M 52S |
| Frequency of IT interventions in the last 7 days | 4M 51S |
| Internet access provided by government at the facility | 4M 51S |
| Responsible for IT maintenance / intervention | 4M 51S |
| Average time driving to ${reffname} | 4M 46S |
| Are there existing mechanisms for recording staff training? | 4M 37S |
| Mechanisms used to directly inform the referral facility of the referral of a child 0-59 months? | 4M 25S |
| Tests routinely perfomed | 4M 12S |
| Electricity availability in past 7 days (from main or backup source) during facility opening hours | 4M 2S |
| Number of computers available at the facility | 4M 2S |
| Number of computers seen during the assessment | 4M 2S |
| Tasks for which tablets are used? | 4M 2S |
| Growth charts available | 3M 57S |
| Guidelines for Infection Prevention and Control | 3M 57S |
| Height board available | 3M 57S |
| IMCI guidelines | 3M 57S |
| Malaria testing available | 3M 56S |
| Checklists and / or job aids for IMCI | 3M 54S |
| Child weighing scale (250g gradation) available | 3M 54S |
| Emergency Triage Assessment and Treatment (ETAT) guidelines | 3M 54S |
| Guidelines for Basic Emergency Obstetric and Newborn Care (BEmONC) | 3M 54S |
| Guidelines for Comprehensive Emergency Obstetric and Newborn Care (CEmONC) | 3M 54S |
| IMCI guidelines on care of sick young infant 0-2 months | 3M 54S |
| Infant weighing scale (100g gradation) available | 3M 54S |
| Measuring tape available | 3M 54S |
| Oxygen therapy guidelines / job aids | 3M 54S |
| Pulse oximetry guidelines | 3M 54S |
| Stadiometre available | 3M 54S |
| Thermometer available | 3M 54S |
| Consultation records management system? | 3M 42S |
| Registration records management system? | 3M 42S |
| By what mechanisms are you informed by the referral facility? | 3M 30S |
| Information received back on outgoing referrals from referral facility? | 3M 30S |
| Please specify (Mechanisms used to directly inform the referral facility of the referral of a child 0-59 months?) | 3M 30S |
| Do you refer children who require oxygen somewhere else? | 3M 29S |
| Do you refer young infants and neonates somewhere else? | 3M 29S |
| Does the facility centrally record information about outgoing referrals? | 3M 29S |
| Mechanisms used to inform caregiver about the referral of a child 0-59 months | 3M 29S |
| Where do you in priority send children 0-59 months with severe illness who need to be referred? | 3M 29S |
| When was the last time a data quality check was undertaken? | 3M 5S |
| Mechanisms used to inform caregivers about recommended follow-up of children 0-59 months | 2M 57S |
| Diagnostic testing at the facility (including any rapid diagnostic testing) | 2M 40S |
| What information does the facility centrally record during follow-up visits? | 2M 40S |
| Ultrasound on-site available for children 0-59 months | 2M 24S |
| X-ray on-site available for children 0-59 months | 2M 24S |
| At any time during the past 7 days has Haemocue been unavailable (for any reason)? | 2M 20S |
| At any time during the past 7 days has malaria blood film microscopy been unavailable (for any reason)? | 2M 20S |
| At any time during the past 7 days has malaria RDT been unavailable (for any reason)? | 2M 20S |
| Haemoglobin testing / estimation | 2M 20S |
| location | 2M 8S |
| HIV testing | 2M 3S |
| Pocketbook for children | 2M 0S |
| Toilet accessible (unlocked or key available) | 1M 58S |
| Type of toilet | 1M 58S |
| At any time during the past 7 days has blood glucose been unavailable (for any reason)? | 1M 54S |
| staff_childcare | 1M 34S |
| staff_time | 1M 34S |
| At any time during the past 7 days has pulse oximetry been unavailable in the outpatient department for any reason? | 1M 28S |
| Countertop available | 1M 23S |
| Fingertip available | 1M 23S |
| Handheld device available | 1M 23S |
| Paediatric probe available | 1M 23S |
| Type of oxygen supply available | 1M 23S |
| Young infant probe available | 1M 23S |
| staff_training | 1M 19S |
| Does the facility centrally record outcomes of outgoing referrals? | 1M 18S |
| At any time during the past 7 days has stool testing been unavailable (for any reason)? | 1M 15S |
| At any time during the past 7 days has urine testing been unavailable (for any reason)? | 1M 15S |
| Stool analysis | 1M 15S |
| Urine | 1M 15S |
| Please select the current district | 1M 3S |
| At any time during the past 7 days has HIV dried blood spot collection been unavailable (for any reason)? | 57S |
| At any time during the past 7 days has HIV rapid testing been unavailable (for any reason)? | 57S |
| At any time during the past 7 days has TB testing been unavailable (for any reason)? | 57S |
| staff_type | 57S |
| TB testing | 57S |
| Number of departures among the clinical staff in the last 3 months | 54S |
| Number of arrivals among the clinical staff in the last 3 months | 50S |
| Thermometer functional | 46S |
| cmpl_imci_training | 16S |
| path_imci_training | 16S |
| staff_status | 10S |
| staff_end | 5S |
5.2 Count of Input Changes and Median Time until Input was Changed by Question
df_changes_per_question <- df %>%
filter(event=="question",
!is.na(time_till_change)) %>%
group_by(question_decoded) %>%
summarise(count_input_changes=n(),
median_time_till_change=median(time_till_change),
sd_time_till_change=sd(time_till_change)) %>%
arrange(desc(count_input_changes)) %>%
mutate(median_time_till_change = round(seconds_to_period(median_time_till_change)),
sd_time_till_change = round(seconds_to_period(sd_time_till_change), 1)) %>%
filter(count_input_changes > 1)
df_changes_per_question| question_decoded | count_input_changes | median_time_till_change | sd_time_till_change |
|---|---|---|---|
| staff_end | 17 | 2S | 13.8S |
| NA | 4 | 3S | 9S |
| Among registers available at the facility, which one(s) are used for HMIS outpatient children 0-59 months reporting? | 3 | 3S | 12.4S |
| Ampicillin | 3 | 3S | 10.4S |
| Artemether injection | 3 | 3S | 0.6S |
| At any time during the past 7 days has pulse oximetry been unavailable in the outpatient department for any reason? | 3 | 1S | 2.3S |
| Ciprofloxacin | 3 | 3S | 0.6S |
| Follow-up visits reported to HMIS? | 3 | 3S | 10.4S |
| For this health facility, who makes decisions about the overall staffing structure and staffing resources (how many providers and staff are hired to work here, the types of staffs are hired to work here etc.)? | 3 | 3S | 2S |
| How do external supervisors monitor the quality of services at this facility? | 3 | 3S | 11.6S |
| How much time is taken from the facility staff to report facility data related to outpatient children 0-59 months per week? | 3 | 3S | 4S |
| Individual clinical notes | 3 | 3S | 1.2S |
| Information received back on outgoing referrals from referral facility? | 3 | 3S | 1.2S |
| Mechanisms used to inform caregivers about recommended follow-up of children 0-59 months | 3 | 3S | 1.2S |
| Personal protective equipment | 3 | 3S | 1.2S |
| staff_status | 3 | 4S | 14.4S |
| Stock outs in the last 7 days | 3 | 5S | 8.1S |
| Available modes of communication | 2 | 3S | 0S |
| Azithromycin | 2 | 3S | 0S |
| Diazepam | 2 | 3S | 0S |
| Does the facility centrally record information about outgoing referrals? | 2 | 3S | 0S |
| Electricity availability in past 7 days (from main or backup source) during facility opening hours | 2 | 3S | 0S |
| For this health facility, who participates in the budget planning and budget development process? | 2 | 4S | 1.4S |
| For what population is the facility policy to charge no fees? | 2 | 2S | 1.4S |
| Gentamicin | 2 | 3S | 0S |
| Guidelines for Basic Emergency Obstetric and Newborn Care (BEmONC) | 2 | 3S | 2.8S |
| Guidelines for Comprehensive Emergency Obstetric and Newborn Care (CEmONC) | 2 | 2S | 1.4S |
| Is there a system at the facility to keep track of staff trainings and certifications? | 2 | 4S | 1.4S |
| Is there any mechanism to ensure the accuracy of recorded information on registries? | |||
| OBTAIN documents related to data quality checks (reports, feedback, checklists). | 2 | 2S | 1.4S |
| Mechanisms used to directly inform the referral facility of the referral of a child 0-59 months? | 2 | 3S | 0S |
| NG tubes | 2 | 3S | 0S |
| Oxygen therapy guidelines / job aids | 2 | 3S | 2.8S |
| Pulse oximetry guidelines | 2 | 3S | 2.8S |
| Quinine (injection) | 2 | 2S | 1.4S |
| Quinine (oral) | 2 | 3S | 0S |
| Registers / ledger recording available at the facility | 2 | 2S | 0.7S |
| Salbutamol | 2 | 10S | 11.3S |
| Source of supplies / consumables / equipment for this facility? | 2 | 4S | 1.4S |
| Stock replenishment mechanism | 2 | 12S | 13.4S |
| What is the procedure if a caregiver is unable to pay for any of the fees associated with healthcare provided to a sick child 0-59 months in this facility? | 2 | 4S | 1.4S |
5.3 Count of Old-New Value Pairs
df_stream <- df %>%
filter(!is.na(time_till_change)) %>%
count(question_decoded,
old_value_decoded,
new_value_decoded,
name="count_value_pairs",
sort=TRUE) %>%
filter(count_value_pairs > 1)
df_stream| question_decoded | old_value_decoded | new_value_decoded | count_value_pairs |
|---|---|---|---|
| staff_end | No | Yes | 11 |
| staff_end | Yes | No | 6 |
| Electricity availability in past 7 days (from main or backup source) during facility opening hours | Always available (no interruptions) | Often available (interruptions <2hrs / day) | 2 |
| Follow-up visits reported to HMIS? | Yes | No | 2 |
| For what population is the facility policy to charge no fees? | Children [0-59 months] with health insurance only | All children [0-59 months] | 2 |
| Gentamicin | O+ | N+ | 2 |
| Guidelines for Basic Emergency Obstetric and Newborn Care (BEmONC) | R | N | 2 |
| Individual clinical notes | Copy held by facility (informal system) | Copy held by facility (formal records system), Copy held by caregiver (formal child health card) | 2 |
| Is there a system at the facility to keep track of staff trainings and certifications? | Yes | No | 2 |
| Oxygen therapy guidelines / job aids | R | N | 2 |
| staff_status | Assigned | Employed | 2 |
| What is the procedure if a caregiver is unable to pay for any of the fees associated with healthcare provided to a sick child 0-59 months in this facility? | Service not provided, asked to come back when able to pay | Other | 2 |
6 Aggregated by Instance
6.1 Top 10 % of Duration by Instance
df_duration_per_inst <- df %>%
group_by(`instance ID`) %>%
summarise(duration_per_inst = max(end, na.rm=T) - min(start, na.rm=T)) %>%
filter(duration_per_inst>quantile(duration_per_inst, 0.9, na.rm=TRUE)) %>%
mutate(duration_per_inst = round(seconds_to_period(duration_per_inst))) %>%
arrange(desc(duration_per_inst))
df_duration_per_inst| instance ID | duration_per_inst |
|---|---|
| uuid:f505d065-163e-42a3-8a85-25988890b584 | 4d 23H 44M 27S |
| uuid:421e5faf-9109-49da-a8f6-c2059d496391 | 3d 1H 51M 43S |
6.2 Distribution of Duration by Instance with Top 10 % excluded
df_subsetted <- df %>%
group_by(`instance ID`) %>%
summarise(duration_per_inst = max(end, na.rm=T) - min(start, na.rm=T)) %>%
filter(duration_per_inst<quantile(duration_per_inst, 0.9, na.rm=TRUE))
hist(as.numeric(df_subsetted$duration_per_inst/60), breaks=30, main="Duration per Instance in Minutes (outliers removed)", xlab="Duration in Minutes")
7 Irregularities and Outliers
7.1 Time Till Change Outliers (for all data without removed outliers)
df_time_till_change_outliers <- df %>%
filter(time_till_change>quantile(df$time_till_change, 0.9, na.rm=TRUE)) %>%
arrange(desc(time_till_change)) %>%
mutate(time_till_change = round(seconds_to_period(time_till_change))) %>%
select(`instance ID`,
question_decoded,
old_value_decoded,
new_value_decoded,
time_till_change)
df_time_till_change_outliers| instance ID | question_decoded | old_value_decoded | new_value_decoded | time_till_change |
|---|---|---|---|---|
| uuid:421e5faf-9109-49da-a8f6-c2059d496391 | staff_end | No | Yes | 49S |
| uuid:49a885da-765e-48b4-9fbd-c60b76c0c640 | staff_end | No | Yes | 35S |
| uuid:b4ff7422-1179-4fa6-aef5-8f3773b70854 | Among registers available at the facility, which one(s) are used for HMIS outpatient children 0-59 months reporting? | Consultation, Laboratory, Pharmacy | Consultation, Laboratory, Pharmacy, Registration | 24S |
| uuid:b4ff7422-1179-4fa6-aef5-8f3773b70854 | Stock replenishment mechanism | As required ordering by facility | Periodic delivery (not as a result of ordering) | 22S |
| uuid:2925ca93-365f-40e7-9107-0d414c60c115 | Deliveries at the facility (sub-categories if C-section vs other available) reported to HMIS? | Yes | No | 21S |
| uuid:2925ca93-365f-40e7-9107-0d414c60c115 | Follow-up visits reported to HMIS? | Yes | No | 21S |
| uuid:6f2f5d66-022c-4a0d-ad0d-ecf0e827e82f | NA | 1 | 0 | 21S |
| uuid:6f2f5d66-022c-4a0d-ad0d-ecf0e827e82f | Ampicillin | N+ | N- | 21S |
| uuid:49a885da-765e-48b4-9fbd-c60b76c0c640 | staff_end | No | Yes | 19S |
| uuid:b4ff7422-1179-4fa6-aef5-8f3773b70854 | staff_end | No | Yes | 19S |
| uuid:421e5faf-9109-49da-a8f6-c2059d496391 | Salbutamol | N+ | O+ | 18S |
| uuid:421e5faf-9109-49da-a8f6-c2059d496391 | Stock outs in the last 7 days | Gentamicin | Co-amoxiclav | 18S |
| uuid:49a885da-765e-48b4-9fbd-c60b76c0c640 | staff_end | No | Yes | 12S |
| uuid:fe61cd8c-5304-4375-98f5-3381e90e75a5 | Number of arrivals among the clinical staff in the last 3 months | 0 | 1 | 12S |
| uuid:49a885da-765e-48b4-9fbd-c60b76c0c640 | staff_end | No | Yes | 10S |
| uuid:6f2f5d66-022c-4a0d-ad0d-ecf0e827e82f | How much time is taken from the facility staff to report facility data related to outpatient children 0-59 months per week? | 48 | 72 | 10S |
7.2 Histograms of Instances with Inconsistent Filling Behaviour
irregular_inst = c()
for (id in unique(df$`instance ID`)){
bin_vec = cut(df$start[df$`instance ID`==id],
breaks=10,
labels=F)
if (length(unique(bin_vec)) < 5) irregular_inst = c(irregular_inst, id)
}
paste0(length(irregular_inst), " out of ", length(unique(df$`instance ID`))," instances were found to have an inconsistent filling behaviour.")## [1] "10 out of 14 instances were found to have an inconsistent filling behaviour."last_bin_questions = c()
fig <- plot_ly(alpha=0.1)
for (id in irregular_inst){
temp_df = df[df$`instance ID`==id,]
temp_df$cut = cut(temp_df$start, breaks=10, labels=c("1. Part", "2. Part", "3. Part", "4. Part", "5. Part", "6. Part", "7. Part", "8. Part", "9. Part", "10. Part"))
fig <- fig %>% add_histogram(x=temp_df$cut, name=id)
last_bin_questions = c(last_bin_questions, temp_df$question_decoded[temp_df$cut=="10. Part"])
}
fig <- fig %>% layout(barmode = "overlay")
figkable(table(last_bin_questions) %>% as.data.frame() %>% arrange(desc(Freq)))| last_bin_questions | Freq |
|---|---|
| staff_type | 7 |
| How much time is taken from the facility staff to report facility data related to outpatient children 0-59 months per week? | 6 |
| staff_end | 6 |
| Among registers available at the facility, which one(s) are used for HMIS outpatient children 0-59 months reporting? | 5 |
| Follow-up visits reported to HMIS? | 5 |
| How do external supervisors monitor the quality of services at this facility? | 4 |
| Individual clinical notes | 4 |
| New visits reported to HMIS? | 4 |
| Referrals reported to HMIS? | 4 |
| Registers / ledger recording available at the facility | 4 |
| Artemether injection | 3 |
| Ciprofloxacin | 3 |
| Deliveries at the facility (sub-categories if C-section vs other available) reported to HMIS? | 3 |
| Diagnoses reported to HMIS? | 3 |
| Did the supervision visit result in the following outputs? | 3 |
| For this health facility, who decides which services are available/on offer to clients? | 3 |
| For this health facility, who makes decisions about the overall staffing structure and staffing resources (how many providers and staff are hired to work here, the types of staffs are hired to work here etc.)? | 3 |
| For this health facility, who participates in the budget planning and budget development process? | 3 |
| How are areas for quality improvement in relation to clinical service delivery identified / agreed? | 3 |
| How are possible solutions to identified areas for quality improvement discussed? | 3 |
| How is implementation of solutions to identified areas for quality improvement supervised? | 3 |
| How is implementation of solutions to identified quality improvement needs monitored? | 3 |
| Is there a system at the facility to keep track of staff trainings and certifications? | 3 |
| Is there any mechanism to ensure the accuracy of recorded information on registries? | |
| OBTAIN documents related to data quality checks (reports, feedback, checklists). | 3 |
| Number of sick children per month reported to HMIS | 3 |
| Source of supplies / consumables / equipment for this facility? | 3 |
| staff_status | 3 |
| What did the supervisor do during the last visit? | 3 |
| What is the procedure if a caregiver is unable to pay for any of the fees associated with healthcare provided to a sick child 0-59 months in this facility? | 3 |
| When was the last time a data quality check was undertaken? | 3 |
| Ampicillin | 2 |
| Are fees charged for the following services for healthcare provided to a sick child 0-59 months ? | 2 |
| Available modes of communication | 2 |
| Azithromycin | 2 |
| By what mechanisms are you informed by the referral facility? | 2 |
| Check completeness (e.g. blank counts) of (provisional or final) diagnoses over the last 20 entries | 2 |
| Check readability of (provisional or final) diagnoses over the last 20 entries | 2 |
| Check relevance of (provisional or final) diagnoses over the last 20 entries | |
| (e.g. check diagnoses only, not symptoms are reported as diagnoses) | 2 |
| Clinical observations recorded in the registration or consultation ledger | 2 |
| Diazepam | 2 |
| Did the last supervision visit result in the planning of the following activities? | 2 |
| Do the funds or resources available to the facility depend on the facility’s past or current performance against specific metrics or goals (either overall or for specific designated targets or goals)? | 2 |
| Does the facility centrally record information about outgoing referrals? | 2 |
| For this health facility, is the budget based on categories of types of expenses (such as staffing, medicines, equipment etc.) or based on categories of the types of services (eg. HIV services, MCH services etc.), or both? | 2 |
| For what population is the facility policy to charge no fees? | 2 |
| Gentamicin | 2 |
| How many external supervision visits have occurred during the 12 last months? | 2 |
| How many healthcare providers receive supportive supervision? | 2 |
| Information received back on outgoing referrals from referral facility? | 2 |
| Laboratory records management system? | 2 |
| Laboratory register / ledger recording specific for children 0-59 months | 2 |
| Mechanisms used to directly inform the referral facility of the referral of a child 0-59 months? | 2 |
| Mechanisms used to inform caregivers about recommended follow-up of children 0-59 months | 2 |
| NG tubes | 2 |
| Pharmacy records management system? | 2 |
| Pharmacy register / ledger recording specific for children 0-59 months | 2 |
| Quinine (oral) | 2 |
| staff_time | 2 |
| staff_training | 2 |
| Stock outs in the last 7 days | 2 |
| Stock replenishment mechanism | 2 |
| Tasks for which computers are used? | 2 |
| Total number of children under 5 consultations reported to HMIS? | 2 |
| Total number of subgroup of # young infant/neonatal consultations reported to HMIS | 2 |
| Triage for children 0-59 months | 2 |
| When was the last time the facility committee happened? | 2 |
| When was the last time this facility received an external supervision visit? | 2 |
| Who is responsible for the day to day management of the facility? | 2 |
| Among those services, which ones are also provided to children 0-2 months? | 1 |
| Amoxicillin - dispersible tablets (500mg) | 1 |
| Amoxicillin - syrup / suspension (250 mg) | 1 |
| Artesunate (rectal) | 1 |
| At any time during the past 7 days has Haemocue been unavailable (for any reason)? | 1 |
| At any time during the past 7 days has malaria blood film microscopy been unavailable (for any reason)? | 1 |
| At any time during the past 7 days has malaria RDT been unavailable (for any reason)? | 1 |
| At any time during the past 7 days has pulse oximetry been unavailable in the outpatient department for any reason? | 1 |
| Chlorhexidine | 1 |
| Cloxacillin | 1 |
| cmpl_imci_training | 1 |
| Co-amoxiclav (Amoxicillin + Clavulanic acid) | 1 |
| Co-trimoxazole syrup / suspension | 1 |
| Communicable disease services | 1 |
| Consultation records management system? | 1 |
| Days of the week for which there are dedicated vaccination / growth monitoring services | 1 |
| Days per week facility for which there is normally at least one eligible healthcare professional is available to consult children 0-59 months? | 1 |
| Does someone else consult if the eligible provider is unavailable? | 1 |
| Does the facility centrally record outcomes of outgoing referrals? | 1 |
| Does this facility have budget earmarked for staff specifically hired for and designated to provide services for infants and/or children exclusively (and not adults)? | 1 |
| Does this facility receive any external supervision? | 1 |
| Does this health facility have any current goals or initiatives for quality improvement in relation to clinical service delivery? | 1 |
| Electricity availability in past 7 days (from main or backup source) during facility opening hours | 1 |
| For this facility, is there a budget or allocation of funds specifically designated to be used for services, staffing, medicines or equipment for infant and or child health curative services (and not other services such as PMTCT, HIV care and treatment or adult services)? | 1 |
| Frequency of facility committees | 1 |
| Gentian violet | 1 |
| Guidelines for Comprehensive Emergency Obstetric and Newborn Care (CEmONC) | 1 |
| Haemoglobin testing / estimation | 1 |
| Height board available | 1 |
| How frequently is data quality checked? | 1 |
| Is the emergency department separate from the outpatient area? | 1 |
| Malaria testing available | 1 |
| Mebendazole Tablet | 1 |
| Neonatal IV needles | 1 |
| Nitrofurantoin | 1 |
| Number of days for which there was no eligible healthcare provider available to consult children 0-59 months in the last 7 days? | 1 |
| Oral artemisinin combination therapy (paediatric dose) | 1 |
| Paediatric IV needles | 1 |
| Paracetamol syrup/suspension | 1 |
| path_imci_training | 1 |
| Personal protective equipment | 1 |
| Primaquine | 1 |
| Quinine (injection) | 1 |
| Registration records management system? | 1 |
| Rifampicin / Isoniazid / Pyrazinamide / Etambutol (RHZE) | 1 |
| Salbutamol | 1 |
| Segregation of waste | 1 |
| Services provided to children 2-59 months | 1 |
| Source of revenue or funding of this facility? | 1 |
| staff_childcare | 1 |
| Tests routinely perfomed | 1 |
| Toilet functional | 1 |
| Training log management system | 1 |
| Was IMCI part of the supervision visit? | 1 |
| What did the supervisor observe during the last visit? | 1 |
| What happens when the internet monthly data limit is reached? | 1 |
| What information does the facility centrally record during follow-up visits? | 1 |
| What quality improvement goals or initiatives in relation to clinical service delivery are currently ongoing in this facility? | 1 |
| Where activities were identified, were they followed upon as part of supervision? | 1 |
| Where do you refer children 0-59 months who require oxygen then? | 1 |